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Evolution of the magnetic response function in the triangular-lattice Hubbard 
model is studied with interaction strength within a systematic inverse-degeneracy 
expansion scheme which incorporates self-energy and vertex corrections and explic- 
itly preserves the spin-rotation symmetry. It is shown that at half filling the response 
function goes through a nearly dispersionless regime around K for intermediate cou- 
pling strength, before undergoing an inversion at strong coupling, resulting in max- 
imum response at the K point, consistent with the expected 120° AF instability. 
Effects of finite hole/electron doping on the magnetic response function are also 
examined. 
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I. INTRODUCTION 

There has been renewed interest in correlated electron systems on triangular lattices, as 
evidenced by recent studies of antiferromagnetism, superconductivity and metal-insulator 
transition in the organic systems n — (BEDT — TTF)2X,i'^ the discovery of superconduc- 
tivity in NaxCo02.yH20,~ the observation of low-temperature insulating phases in some 
v^-adlayer structures such as K on Si[lll],^ and quasi two-dimensional 120° spin order- 
ing and spin-wave excitations in RbFe(Mo04)2 (Refs. 5,6) and the multiferroic materials 
YMnOa and HoMnOgl'^'^'i^ 

Transverse spin fluctuations in the 120° ordered antiferromagnetic (AF) state of the 
triangular-lattice Hubbard model at half filling were recently investigated in the full range of 
interaction strength U.—^ While a stable AF state was obtained in the strong-coupling limit, 
with identical spin-wave dispersion as for the equivalent QHAF, with decreasing U the spin 
stiffness was found to vanish at f/ ^ 6. The loss of magnetic order due to divergent quantum 
fluctuations yields a magnetic phase transition to a quantum spin-disordered insulator, which 
is relevant to the spin-liquid state and Mott transition in the organic systems k — (BEDT — 
TTF)2X. The vanishing spin stiffness implies that the triangular-lattice Hubbard model 
exhibits, besides the intrinsic geometrical frustration of the triangular lattice, an additional 
[/-controlled frustration due to competing extended-range spin couplings generated at finite 
U. The existence of stable 120° AF ordering at large U, but the vanishing spin stiffness (as 
well as energy ujm) indicates competition with other magnetic orderings with decreasing U. 

In view of this enhanced magnetic frustration in the triangular-lattice Hubbard model at 
finite U, it is therefore of interest to examine how the magnetic response function evolves 
with increasing interaction strength in the correlated paramagnetic state. Even at the bare 
level, the magnetic response function shows very rich behaviour [Fig. 1]. The comparable 
magnetic response at different symmetry points in the Brillouin zone, correponding to very 
different magnetic orderings, represents the weak-coupling picture of competing orders and 
magnetic frustration in the triangular-lattice paramagnet. Furthermore, the bare response 
is not maximum at the K point corresponding to 120° AF ordering. In view of the expected 
instability towards the 120° AF ordering at strong coupling, it will be desirable to develop 
an approach wherein the evolution of the magnetic response with increasing interaction 
strength U is consistent with this AF instability. 
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Given that the 120° AF state is stable in the strong-couphng hmit, it would be desirable 
to use a many-body approach which continuously interpolates to the spontaneously-broken- 
symmetry AF state and yields a proper description of the Goldstone mode and spin waves by 
preserving the spin- rotation symmetry. From this viewpoint, evaluating many-body correc- 
tions reliably in the intermediate and strong-coupling regimes remains a challenge. Schemes 
such as the dynamical mean-field theory (DMFT) and fiuctuation-exchange (FLEX) approx- 
imation, although providing powerful tools for studying the correlated paramagnet, do not 
continue into the broken-symmetry state without breaking the essential spin-rotation sym- 
metry, while the two-particle self-consistent (TPSC) approximation does not the renormalize 
the momentum structure of the magnetic response function. 

In this paper we will use a systematic many-body expansion scheme to investigate the 
momentum-dependent magnetic response function, and study its evolution with increasing 
interaction strength. For this purpose we will use a systematic inverse-degeneracy (l/A/") 
expansion scheme which explicitly preserves the spin-rotational symmetry by including self- 
energy diagrams as well as the corresponding vertex corrections. The importance of in- 
cluding vertex corrections in preserving spin-rotation symmetry has been highlighted in the 
context of paramagnon corrections^"^ in He^, the antiferro magnetic ground state, and the 
ferromagnetic ground state.— Indeed, we will show that the dominant quantum correc- 
tion to the response function arises from the vertex corrections, signifying suppression due to 
particle-particle correlations. Therefore, from the paramagnetic side, vertex corrections play 
a dominant role in suppressing magnetic frustration and stabilizing the 120° AF ordering at 
half filling. 

The Hubbard model on a triangular lattice has been studied recently using a variety of 
tools. The zero-temperature phase diagram has been studied using the slave boson (SB) 
technique and the exact diagonalization.— The mean-field SB approach yields a rich phase 
diagram qualitatively resembling the Hartree-Fock results.—"^ The non-magnetic insulating 
state near the Mott transition has been studied using the path integral renormalization 
group method,— in which the HF results are systematically improved to reach the true 
ground state by taking account of quantum fiuctuations. Results show a generic emergence 
of a non-magnetic insulating state sandwiched by a Mott metal-insulator transition and an 
AF transition. One-electron density of states has been examined using the quantum Monte 
Carlo method,— showing a pseudogap development for intermediate U, accompanied by two 
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peaks in the spin structure factor, signaling the formation of a spiral spin density wave 
(SDW). A weak-coupling RG analysis applied to the anisotropic triangular lattice shows 
that frustration suppresses the antiferromagnetic instability in favour of a superconduct- 
ing instability.^^ A magnetic field induced exotic spin-triplet superconductivity has been 
proposed having strong ferromagnetic fluctuations.— 

A spin-liquid type non-magnetic insulating (NMI) state sandwiched between a weak- 
coupling PM state and a strong-coupling AFI state has also been obtained for the t — t'- 
Hubbard model on a square lattice and an anisotropic triangular lattice using the path 
integral renormalization group method.—*^ The NMI state has been recently suggested to 
be a new type of degenerate quantum spin phase having gapless and dispersionless spin 
excitations.~ At the same time, this result of an intervening NMI state is in contradiction 
to the earlier finding of an intermediate metallic antiferromagnetic state (AFM).— In the 
context of (BEDT — TTF)2Cu2(CN)3, spin-liquid phases near the Mott transition in the 
Hubbard model have also been studied within the U(l) gauge theory.— 

The organization of the paper is as follows. The inverse-degeneracy expansion scheme is 
briefly reviewed in section II. The order-l/A/" diagrams for the irreducible propagator and 
their expressions are given in section III. Results at half filling and for finite electron/hole 
doping are discussed in sections IV and V, and conclusions are presented in section VI. 
Evaluation of the fermion vertices by integrating out the fermion energy-momentum degrees 
of freedom are illustrated in Appendix A. Emergence of the pseudo gap in the one-electron 
density of states due to order-l/A/" self-energy corrections is discussed in Appendix B. 

II. INVERSE-DEGENERACY EXPANSION 

We consider the generalized AT-orbital Hubbard model:— 

H = -t ^ {al^aj^a + H.c.) + ^^{Uial^^ai^^al^^aup + f/2aj-f„ai-f/3aj^^ai|„) , (1) 

{ij),(j,a i,c»,/3 

where refer to the degenerate orbital indices and the factor l/M is included to render 
the energy density finite in the M ^ oo limit. In the isotropic limit {Ui = U2 = U), the 
two interaction terms (density- density and exchange-type with respect to orbital indices) are 
together equal to [/(— Sj.Sj + n^) in terms of the total spin Si = J2a i'iai^/'^)'^ia and charge 
rii = ipl^{l/2)ipia operators, and the Hamiltonian is therefore explicitly spin-rotationally 
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symmetric. 

With z as the spin-quantization direction, it is convenient to evaluate the time-ordered 
transverse spin-fluctuation propagator: 

(q,^) = z /rft e^-(*-*') 5^5^e^''(--r^)(M/o|T[5rJt)5+ (t')]^o) (2) 

involving the spin-lowering and spin-raising operators 5*^ = il)\a^ / 2)il) , where ip is the elec- 
tron field operator. The transverse propagators ^ind yield the x, y components of 
the magnetic response, which are identical to the z response due to spin-isotropy in the para- 
magnetic ground state \^o). When evaluated in the spontaneously-broken-symmetry state, 
the transverse spin-fluctuation propagator also describes collective spin-wave and particle- 
hole Stoner excitations.—*^ 

In terms of the exact irreducible propagator 0(q, cj), which incorporates all self-energy 
and vertex corrections, the spin-fluctuation propagator can be generally expressed as: 

X q,^=i — 77-7 — 7- (3) 

The inverse-degeneracy expansion:— 

= 0^°^+(;^)0«+(;^)%(^^ + - (4) 

systematizes the diagrams for cj) in powers of the expansion parameter l/A/" which, in analogy 
with 1/ S for quantum spin systems, plays the role of h. This expansion explicitly preserves 
spin-rotational symmetry and therefore the Goldstone mode in the broken-symmetry state, 
and has been used recently to evaluate quantum corrections to spin-wave energies and spin 
stiffness in the antiferromagnetio^ and ferromagnetio^^ states of the Hubbard model. 

M ^ CO limit 

In the A/" — i> 00 limit, only the "classical" term 0'^'') = x° survives, and the ladder series 
with interaction U2 yields the random phase approximation (RPA): 

amounting to a classical description of non-interacting spin-fluctuation modes. Here the 
bare antiparallel-spin particle-hole propagator: 

0W(q,c.) ^ /(q,^) = E {— F^^ + — P ) ' (6) 
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FIG. 1: The bare magnetic response function at different fillings, showing the relative suppression 
(enhancement) of the K response (which corresponds to 120° AF ordering) with electron (hole) 
doping. Also shown are few constant-energy contours in the triangular-lattice Brillouin zone. 

where = ek + are the Hartree-Fock (HF) band energies in the paramagnetic state, 

and the superscript +(— ) refers to particle (hole) states above (below) the Fermi energy 
E-p. The spin-independent HF band-energy shift Uin/2, corresponding to the A/" ^ oo self 
energy, can be trivially transformed away by an energy shift, as assumed henceforth. 

The bare particle-hole propagator x'^i'i) yields the magnetic response to a static, spatially- 
varying magnetic field, the rich behaviour of which is shown in Fig. 1 for different Fermi 
energies, with corresponding fillings n = 0.6,0.8,1.0,1.1, respectively. Also shown are few 
constant-energy contours in the triangular-lattice Brillouin zone. Contributions to x° from 
particle states near the nested hexagonal contour (ek = 2) with divergent density of states 
is responsible for the cusps in the bare magnetic response function. For half filling, the 
most significant features are the comparable response at the three symmetry points F, M, 
K, and peaks at the three points approximately midway between them. The magnetic 
orderings corresponding to the three symmetry points F, M, K are (0,0,0), (0,7r,7r), and 
(27r/3, 27r/3, 27r/3), respectively, where the triplet corresponds to the ordering wave vector 
in the three lattice directions. Similarly, for the three midpoints M/2, (M+K)/2, 3K/4, the 
ordering wave vectors are (0, 7r/4, 7r/4), (vr/G, 77r/6, Tvr/G), and (tt, 7r/4, 7r/4), respectively. 

The comparable magnetic response at these six symmetry points, which correpond to 
very different magnetic orderings, represents the weak-coupling picture of competing orders 
and magnetic frustration in the triangular-lattice paramagnet. Note that the response at K, 
corresponding to 120° ordering, is not maximum. In view of the expected instability towards 
120° AF ordering at strong coupling, it is of particular interest to examine the evolution of 
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FIG. 2: The first-order quantum corrections to the irreducible particle-hole propagator (/>(q, cj). 
the magnetic response with increasing interaction U. 
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III. l/J\f CORRECTIONS 



The order- diagrams which yield quantum correction 0^^^ to the irreducible particle- 
hole propagator 0(q, cu) are shown in Figure O The fermion lines represent the HF Green's 
functions G". Diagrams (61) and (62) involve self-energy corrections due to transverse (ladder 
diagrams) and longitudinal (bubble diagrams) spin fluctuations, respectively. Other self- 
energy diagrams with l/M corrections to the Hartree self energy, as in the antiferromagnetic 
state, vanish identically as there are no quantum corrections to particle densities in the 
paramagnetic state. Diagrams (a) and (c) involve vertex corrections, with (c) representing 
coupling of longitudinal and transverse spin fluctuations. 

The quantum corrections represented by diagrams (a), (b), (c) incorporate different as- 
pects of correlation effects. These include renormalized and dynamical effective interaction 
(a), negative correction due to spectral- weight transfer and energy renormalization arising 
from self-energy corrections (b), and negative contribution due to particle-particle correla- 
tions of the crossed diagrams (c2). The strong particle-particle correlations found in this 
study, which suppress the magnetic response through the vertex corrections, are relevant 
for pairing correlations in the context of the observed superconductivity in the BEDT com- 
pounds. 

The corresponding expressions are given below. For diagram (a), we obtain: 




(7) 



where the four-fermion vertex: 
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and the effective antiparallel-spin interaction: 




(Q,fi) 



(9) 



involves the even-bubble series with interaction Ui. 



For diagrams {hi) and (62) involving self-energy corrections, we obtain: 




(10) 
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where the four-fermion vertex: 



^ ^[G'°(k',cu')]'G^°(k' - Q, w'-f])[G'°(k' - q,(^'-cu)+G°(k' + q,cu'+^)], (li; 



and the effective interaction 

includes the transverse contribution U~ff'{Q,Q), involving the RPA ladder series (with in- 
teraction U2) and the parallel-spin contribution U^^{Q,Q), involving the RPA odd-bubble 
series (with interaction Ui). 

Finally, for the vertex correction diagrams (ci) and (C2), involving both the ladder series 
(with interaction U2) and the bubble series (with interaction Ui), we obtain: 

(12) 

f^J]27(^H(Q,Q)7(^)-(Q,Q) 

(l-t/Xo(Q,^^)) (l-t/2xg(q-Q,^-^^)) ' ^^^^ 
where the three-fermion vertices: 

/oo J / 
— y G°(k' ± q, a;' ± a;)G'°(k', a;')G'°(k' d= Q, a;' ± 1^) . (14) 

It is straightforward to show, using transformations such as (k', uj') — > (k' + q, a;' + uj) 

and k', Q — k', — Q, that the quantum corrections are symmetric in the q, a; 
space: 

0(-q, -uj) = 0(q, uj) = 0(-q, a;). (15) 



IV. RESULTS AT HALF FILLING 



In this section we present results at half filling for the order-l/jV contributions 
in the static limit and discuss the evolution of the static magnetic response with increasing 
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FIG. 3: The 0-resolved contributions of (^('^^^ (f)^'^^\ and (/>("\ showing enhanced contributions at 
low frequency (paramagnon enhancement) as well as at intermediate and high frequencies. 
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FIG. 4: Comparison of (f)^'^^ with the second-order result, with respect to the fi-resolved contribution 
(a) and the q dependence (b). 

interaction strength U. Evaluation of the fermion energy-momentum integrals for the three- 
and four- fermion vertices (Eqs. 8,11,14), corresponding to integrating out the fermion de- 
grees of freedom, is illustrated in Appendix A. In our numerical calculations for (;/)('^)'(^)'(^) we 
have taken grid sizes dk' = dQ = 0.1, dQ = 0.2, and r] = 0.1, on an energy scale t = 1. 

In order to examine the relative contributions to the three quantum corrections {;/)(")'('')'('^) 
from the different internal bosonic modes (involving ladders and bubbles) and the three- 
and four-fermion vertices 7*^^^ and we introduce functions T^"-^'^''^''^'^\Q) defined by: 



1 poo 
(a),(fe),(c) ^ ±_ 

w 



(16) 



where W is the fermion bandwidth. The functions T{Q) effectively yield combined density 
of states of the internal excitations involving the vertex functions and the bosonic modes. 
Typical plots for r{Q)/W are shown in Fig. [31 The symmetric-fi behaviour provides a 
numerical check for the calculations. It also shows that the intermediate- and high-f2 con- 
tributions are quite significant and comparable to the low-fi contributions which show the 
usual sharp paramagnon enhancement. 

The f2-resolved contribution of cp^^^ with only the leading, second-order (f/^) term in Eq. 
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FIG. 5: Evolution of the different contributions (p^"'^ and (f)^^^ with U, showing enhanced 

negative contribution of (f)^^^ and (f)^'^'^ at the same q points where x^{^) peaks. 



(13) is also shown in Fig. 111(a) for comparison. Apart from the missing low-fi paramagnon 
enhancement as expected, it strongly resembles the f2-structure of the full ^'■'^^^ implying an 
essentially fermionic origin for the intermediate- and high-energy structures. Also shown (b) 
is a comparison of the q dependence of (p^'^^ with the second-order result. The exactly same 
q-structure implies that the fermionic terms 7*^'^-' are fully responsible for the characteristic 
momentum dependence as well. Renormalization of the internal paramagnon mode, within 
a self-consistent approach, is therefore not expected to qualitatively change this q-structure. 

Evolution of the different contributions 0*-"^ cp^''^ and (p'^^^ with increasing interaction 
strength U is shown in Fig. [5l along with their relative comparison for U = 4. An enhanced 
negative contribution of (p^^^ and (p^^^ is seen to occur at the same q points where x^i^) peaks. 
It should also be noted that although the self-energy contribution (b) has similar momentum 
dependence, the largest contribution comes from the vertex correction (c). Furthermore, 
the relative difference between the K and peak contributions are significantly enhanced 
as compared to that in In order to make contact with the single-orbital Hubbard 
model, double counting at the f/^ level in diagrams (c2) and (b) was removed in the Fig. 5 
calculations. 

In order to examine the t/-evolution of the full static magnetic response, we evaluate the 
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irreducible propagator to all orders within an approximate resummation scheme:- 

^'■"^ = T^wwm 

which is exact to first order in the expansion. Fig. El which summarizes the main 

result of this paper, shows the evolution of 0(q) with increasing U. The enhanced negative 
contribution of the quantum corrections 0'-''^ and 0*^^^ — at the same q points where the 
bare response x^i'i) peaks — results in an inversion of the curvature around K and M 
with increasing U. The net response is maximum at K, indicating stabilization of the 120°- 
ordered AF state in the strong-coupling limit. This is consistent with the consensus of 
a 120°-ordered AF ground state for the equivalent 5* = 1/2, nearest-neighbour quantum 
Heisenberg antiferromagnet (QHAF) on a triangular lattice.— '^'^'^'^ 

The change in the curvature of the response function with increasing U implies that it 
goes through a regime of nearly fiat magnetic response around K, as seen in Fig. [61 This is in 
agreement with the observed lack of dispersion in recent PIRG calculations, where the NMI 
state has been suggested to be a new type of degenerate quantum spin phase having gap- 
less and dispersionless (flat) spin excitations, indicating a high momentum degeneracy and 
accounting for the quantum melting of simple translational symmetry breakings including 
the AF long-ranged order.— 

This evolution of the magnetic response function highlights a complex feature of cor- 
relation effects on magnetic frustration in the triangular-lattice Hubbard model. Initially, 
increasing interaction strength results in enhanced competing interactions and magnetic 
frustration, as indicated by the decreasing (negative) curvature of the magnetic response, 
which even becomes flat in a broad momentum range around K, with the degenerate response 
indicating high degree of spin disorder. However, beyond a critical interaction strength, the 
magnetic response develops increasingly positive curvature around K, indicating build-up of 
120° AF spin correlations and suppression of magnetic frustration. In the strong-coupling 
limit, long-range 120° AF order sets in, as only the geometrical frustration of the triangular 
lattice remains due to the surviving NN AF spin couplings in the equivalent QHAF. 

The overall suppression of the magnetic response with increasing f/ is a manifestation 
of correlation effects, arising mainly from the particle-particle correlations involved in the 
vertex correction (c2) with crossed interaction lines and, to a relatively smaller extent, also 
from the pseudo-gap formation due to self-energy corrections (b). From the paramagnetic 
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FIG. 6: Evolution of the magnetic response function (/'(q) with U, showing the inversion of the 
curvature around K and M with increasing U, with maximum response at K indicating stabihzation 
of the 120°-ordered AF state in the strong couphng hmit. 

side, the leading instability towards the 120° AF state at strong coupling and at half filling 
is thus a consequence of these vertex and self-energy corrections. 

The maximum magnetic response at K implies onset of AF spin correlations with short- 
range 120° ordering. What is the effect of these correlations on the self-energy correction 
and the electronic density of states? Using characteristic band dispersion identities for 
the triangular-lattice, an approximate analytical estimate for the electron self energy due 
to spin-fluctuation scattering yields a two-band structure, with similar dispersion as for the 
broken-symmetry state, and band separation increasing with interaction strength, eventually 
leading to the insulating gap.— 

As the renormalized magnetic response is maximum at K, from Eq. (3) we can estimate 
the critical interaction strength U* = l/(f){K) ^ 1/0.14 ^ 7 for the magnetic transition 
to the 120° ordered AF state. In this estimation we have assumed that the renormalized 
mterna/bosonic excitations have the simple form x(Q; ^) = iJu'^^'[qq) ? considered within 
the two-particle self-consistent (TPSC) approximation,— with the renormalized interaction 
f/' ^ 4 corresponding to the maximum bare response x° ~ 0.25. This estimate for U* is 
in good agreement with the value obtained earlier 6) from the broken-symmetry side 
by considering the melting of magnetic order due to quantum spin fluctuations in the 120° 
ordered AF state.— 
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V. FINITE DOPING 

In the preceeding section, the renormahzed magnetic response at half-filhng was shown 
to be maximum at K, in accord with the consensus that 120° AF ordering is stabihzed in 
the strong-coup hug hmit. In this section, we will examine the effects of finite hole and 
electron doping on the magnetic response function, and therefore on the stability of the 120° 
ordered AF state. Earlier studies have shown that the 120° ordered AF state is stabilized for 
hole doping and the spin stiffness is enhanced, whereas it is destabilized for any amount of 
electron doping.— These studies were carried out in the broken-symmetry state, with doped 
holes/electrons introduced in the AF bands within a rigid-band approximation, and effects 
of finite doping on transverse spin fluctuations were examined by including the intraband 
particle-hole processes in the particle-hole propagator. 

Fig. [7] shows a comparison of the bare and renormalized magnetic response for Ep = 
— 1.0 (hole doping) and U = 4. The bare magnetic response shows a pronounced peak 
at K corresponding to 120° AF ordering, indicating drastic suppression of frustration. We 
find that since quantum corrections are significantly suppressed, this feature survives at 
the renormalized level, indicating that the dominant magnetic instability at K remains 
unchanged. 

On the other hand, for electron doping we find a subtle inversion of the magnetic response 
near K on a small-momentum scale, as shown in Fig. 8(a), indicating destabilization with 
respect to long-wavelength fluctuations about the 120° ordering, in agreement with earlier 
results.— We emphasize that although the bare magnetic response also shows a negative 
curvature around K, indicating relative instability of the 120° ordered state, the quantum 
correction introduces a negative curvature on a much smaller momentum scale. Fig. 8(b) 
shows that this small-momentum feature arises from the self energy term in and is 
completely absent in the vertex correction (f)^'^\ 
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FIG. 7: The renormalized magnetic response function </>(q) for U = 4.0 and hole doping {Ep = 
— 1.0) along with the bare response function x^{*i)^ showing that the maximum response remains 
at K, corresponding to 120° AF ordering. 
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FIG. 8: Evolution of 0(q) with U for the electron-doped case shows a dip near K, indicating 
destabilization of the 120° ordered state on a small-momentum scale (a); this small- momentum 
feature near K originates from the self-energy contribution (j)^'^^ and not from the vertex correction 
0W (b). 



VI. CONCLUSIONS 



In conclusion, we have investigated correlation effects on magnetic frustration in the 
triangular-lattice Hubbard model by studying the evolution of the magnetic response func- 
tion with increasing interaction strength. We have employed a systematic inverse-degeneracy 
expansion scheme which preserves spin-rotational symmetry and therefore allows seamless 
interpolation into the broken-symmetry state. 

The strong suppression of the renormalized magnetic response, mainly by vertex correc- 
tions due to particle-particle correlations and to a smaller extent by self-energy corrections 



16 



due to pseudo-gap formation, was shown to result in an inversion of the magnetic response 
function with respect to the bare result, yielding a maximum response at the K point, con- 
sistent with the expected 120° AF instability at half filling and strong coupling. In view of 
the comparable and locally maximum response at M, it is interesting to note that a vr-flux 
spin-liquid state, which on spinon condensation leads to ordering wave vector on the Bril- 
louin zone edge centers (M points), has been proposed for the J — J' QHAF on the triangular 
lattice.-2S 

Hole doping was shown to suppress competing interaction and frustration effects, resulting 
in enhanced stability for 120° AF ordering. On the other hand, for electron doping, a small- 
momentum feature was obtained in the magnetic response function near K corresponding to 
instability of the 120° ordering with respect to long-wavelength fluctuations. These results 
at half filling and finite doping are in agreement with earlier results obtained in the broken- 
symmetry AF state, indicating that this spin-rotationally-symmetric approach can be used 
reliably and seamlessly between the paramagnetic and broken-symmetry states. 

A self-consistent analysis for both the irreducible particle-hole propagator and the self 
energy, incorporating 120° AF spin correlations in the renormalized internal bosonic modes 
(Q,r2), will highlight the effects of these correlations on dynamical features such as the 
spin-fluctuation energy scale (especially for the important low-energy modes), the spectral 
function Imx~+(q, cu) of magnetic excitations (the integrated weight of which yields the 
local spin correlations (S^S^) and local moments in the paramagnetic state), as well as the 
spin-fluctuation self energy and pseudo-gap formation in the electronic density of states. A 
finite-frequency study dealing with these dynamical aspects is currently in progress. 



APPENDIX A: EVALUATION OF THE 3-FERMION VERTEX 

We illustrate here the evaluation of the fermion terms by integrating out the fermion 
energy-momentum degrees of freedom for the three-fermion vertex. Including all possible 
retarded/advanced cases, we obtain: 



7('=)+(Q,fi) = t 



-oo j^, 



°° diu' 



oo 



E 



2n ^ uj' + u- ek,+q ±iv uj' - e^, ±ir] uj' + Vl - ej,,+Q ± ir] 
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k' 



k' 



^k'+q 


- e+ - o; + i77 ■ek,+q - e++Q + - 
1 1 


LU + ir} 




4'+q + ^ + i?7 ' Ck/ - Ck'+Q + + «^ 

1 1 
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1 1 


fl + if] 


^k'+q 


-e^,-uj- ir7'e++q - €^,^^1 + ^ - 
1 1 


UJ — IT] 


4- 


^k'+q + ^^-iv'4'- ^k'+Q + ^-iv 
1 1 




^k'+Q 


- ^k'+q - ^ + ^ - ^^■4'+Q - ^k' - 


fl — if] 



^'T^ ^ ^ • (Al) 

4'+Q - ^k'+q + eJ.+Q -e^,-Q-t7] 

The k' summations were performed numerically over the triangular-lattice Brillouin zone 
with a grid size dk' = 0.1. 



APPENDIX B: 0{l/M) SELF-ENERGY CORRECTION 

In section IV it was mentioned that the negative contribution of (i)'^ arises from the redis- 
tribution of spectral weight due to self-energy corrections. We illustrate here this feature in 
the renormalized density of states resulting from the first-order {l/N) self-energy correction: 

due to exchange of transverse and longitudinal spin fluctuations, corresponding to ladder 
and bubble diagrams, respectively. The two (retarded and advanced) contributions to the 
self energy, 

/•OO JQ -1 

S^(^) = U'Y. — Ini[Xtotai(Q, fl)]K ^ ^— - (B2) 

Q Jo TT U-il- el_(^ + IT] 

and 

— Im[xtotai(Q, ^)]a^7F^ = (B3) 

correspond to the intermediate fermion state k — Q lying inside (— ) or outside (-I-) the 
Fermi surface, and the total spin-fluctuation term [xtotai(Q, ^)] includes both the ladder 
and bubble contributions. The retarded self energy E^(a;) yields negative imaginary part 



xo(Q,^) 



+ 



(Bl) 
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FIG. 9: Evolution of the renormalized density of states evaluated from Eq. (B5), showing the 
opening of the pseudo gap with increasing interaction strength \J . 

only for uj > Ep, whereas the advanced self energy yields positive imaginary part 

only for u < Ep. 

The total self energy T,i^{uj) = + Sj^(ct;) yields the renormalized Green's function: 

1 



and the one-particle density of states: 



N{uj) = - 



TX 



ImSk(u;) 



[cj - ek - ReSk(cu)]2 + [ImSk(cu)]^ 



(B4) 



(B5) 



If the intermediate fermion states k — Q predominantly lie outside (inside) the Fermi 
surface for k inside (outside), as is characteristic of the unfrustrated square lattice for Q 
near the AF ordering wavevector (tt, tt), then the negative (positive) contribution of S^^'^'' {uj) 
pulls down (pushes up) the hole (electron) states in energy, resulting in the opening of an 
energy gap in the one-particle density of states. Fig. [H] shows the emergence of a pseudo gap 
in the renormalized one-particle density of states with increasing interaction strength U . 
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